Skip to content

Conversation

Christopher-Chianelli
Copy link
Contributor

  • Introduces ConstraintFactory.staticData(Uni), which creates an independent NodeNetwork from the provided Uni stream.

  • Introduces TupleSourceRoot that represent top level tuple producers (the ForEach nodes now implement this, as well as a new StaticDataUniNode).

  • The StaticDataUniNode caches the results of its internal NodeNetwork and maps the tuples to new tuples compatiable with the external NodeNetwork

  • Inserts/retracts invalidate the cache and cause the output tuples to be updated.

  • Updates uses the cache and do not notify the internal NodeNetwork.

@triceo
Copy link
Collaborator

triceo commented Oct 6, 2025

This is surprisingly simple! Some high-level comments; leaving naming out of the picture for now:

Introduces ConstraintFactory.staticData(Uni), which creates an independent NodeNetwork from the provided Uni stream.

I'm not a big fan of this. The programming model doesn't look very nice IMO.

factory.staticData(factory.forEach()...)
    .filter(...)

Something like this would be more stream-y:

factory.forEach().
    ...
    .asStatic()
    .filter(...)

This brings its own problems. But both proposals share the same issue - chaining static streams. Nothing (other than run-time fail-fasts) prevents you from doing staticData(staticData(...).filter(...)). IMO we should decide (and enforce) that a static stream can only be built once, and everything after that is dynamic. And ideally, this would happen at type level, not at runtime.

This was the main idea of filterStatic() - because we can actually guarantee, at type level, that once any "non-static" method is called, a "static" method can never be called again. (The reverse doesn't work, unless we want to duplicate the entire stream API, which we don't.) IMO we shouldn't rule out filterStatic() + joinStatic() - under the hood, it can be implemented like in this PR, but the public API IMO need not offer this absolute flexibility to turn anything into static data.


@NullMarked
public interface TupleSourceRoot<A> {
void insert(A a);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure why this used to be @Nullable, but I remember clearly that I was adding it there for a reason.

I recommend adding null checks, running the entire CI incl. quickstarts, and if the null checks don't trigger, then I say it's safe for this to be non-null.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is for neighboorhoods; all the failing tests (when a null check is added to AbstractForEach) are in ai.timefold.solver.core.impl.neighborhood.*. (ChangeMoveDefinition, ListChangeMoveDefinitionTest, UniEnumeratingStreamTest and solveWithNeighborhoodsListVar).

* As this is cached, it is vital the stream does not reference any variables
* (genuine or otherwise).
*/
<A> @NonNull UniConstraintStream<A> staticData(UniConstraintStream<A> stream);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when this stream is used both statically and non-statically?
(Think node-sharing.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output stream can be node shared if the input streams are identical (not implemented currently since this is a POC). The input stream will not be node shared; its an independent network.

Copy link
Collaborator

@triceo triceo Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The input stream may need to be shared though - why build the network twice if the user wants to reuse things?

@triceo
Copy link
Collaborator

triceo commented Oct 6, 2025

To bring an alternative API idea:

 factory.forEachXYZ(Something.class, filter)

This would produce a StaticUniStream which extends UniStream, and only adds one method - joinXYZ(); this join only allows to join other static streams and returns StaticBiStream - allowing possibly for a static bi join (and similarly for tri).

In essence, this brings a very simple, clear API:

  • We support static filters and joins (maybe also ifExists, which is essentially a join).
  • Static operations come first.
  • Once you go dynamic, you can never go back.
  • This is enforced at the type level, preventing impossible situations from being modelled.
  • Node-sharing is trivial; nothing changes in the status quo.

It does have a downside - it doesn't allow arbitrary static streams, such as groupBy, map, flatten etc. Right now, nobody is asking for it, and I think it is a decent trade-off to get the benefits; but we can possibly ask other people for their opinion on this.

Copy link
Collaborator

@triceo triceo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Submitting more comments as I get familiar with the changes.

private final RecordingTupleLifecycle<Tuple_> recordingTupleLifecycle;
private final Class<?>[] sourceClasses;

public <Solution_> BavetStaticDataBuildHelper(BavetAbstractConstraintStream<Solution_> staticConstraintStream) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of this logic is very similar with the other build helper.
Consider sharing code.

}

@Override
public final void settle() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This exposes a confusing API.

Why is there a settle method, and also a propagator which allows external settling? Let's try to find a different solution; clear APIs have only 1 way of doing things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There a strong possibility of a performance regression happening if we introduce a new PropagationQueue type, since then the hot-loop of settle inside the NodeNetwork will have three valid interfaces of Propagator, and JVM method dispatch generate less efficient code when a call site can refer to three or move implementations of a class.

@triceo
Copy link
Collaborator

triceo commented Oct 9, 2025

Good progress! What do you still need to do here?

@Christopher-Chianelli
Copy link
Contributor Author

Implementation should be done; need more tests and a finalized design.

- Introduces ConstraintFactory.staticData(Uni), which creates
  an independent NodeNetwork from the provided Uni stream.

- Introduces TupleSourceRoot that represent top level tuple
  producers (the ForEach nodes now implement this, as well as
  a new StaticDataUniNode).

- The StaticDataUniNode caches the results of its internal
  NodeNetwork and maps the tuples to new tuples compatiable with the
  external NodeNetwork

- Inserts/retracts invalidate the cache and cause the output tuples
  to be updated.

- Updates uses the cache and do not notify the internal NodeNetwork.
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add filterStatic (vs. filter) to ConstraintStreams

2 participants